1. Field of the Invention
The present invention is a method and system for analyzing shopping behavior by associating RFID data, such as tracking data by the RFID tag identifications, with video-based behavior and segmentation data, such as behavior analysis or demographic composition analysis of the customers, in a preferred embodiment, wherein the analyzed shopping behavior helps gain deeper insights in a retail space, using a plurality of means for sensing and using RFID tags, a plurality of means for capturing images, and a plurality of computer vision technologies.
2. Background of the Invention
There have been attempts to use the Radio Frequency IDentification (RFID) technology for tracking non-human objects or persons in the prior art.
U.S. Pat. No. 6,989,741 of Kenny, et al. (hereinafter Kenny) disclosed a method for locating and tracking an object that is attached by an electronic tag, E-Tag, based on the RFID technology and triangulation technology. U.S. Pat. No. 7,053,775 of Moore disclosed a method and apparatus for tracking items automatically using a passive RFID and remote sensing antennas. U.S. Pat. No. 7,091,863 of Ravet disclosed a method and apparatus for tracking the movement and location of an object in a predefined area using at least one readable sensor and at least one reader at a preselected location. U.S. Pat. No. 7,098,793 of Chung disclosed a method and apparatus for tracking objects employing smart tags, which utilized contact-less radio-frequency (RF) transmissions. U.S. Pat. No. 7,116,230 of Klowak disclosed a system for asset location using asset tags, which transmit radio frequency.
Although these prior arts show various approaches of applying the RFID technology to tracking application for various objects, they are clearly foreign to the concept of associating the RFID technology with the computer vision based automatic tracking technology. Furthermore, they are clearly foreign to the idea of associating the RFID technology with the computer vision based demographic composition information of customers, shopping behavior analysis of the customers, and the transaction data in a retail environment.
U.S. Pat. No. 6,154,139 of Heller disclosed a method and apparatus for utilizing both the RFID and infrared (IR) parts of the electromagnetic spectrum to locate subjects in a tracking environment. Heller is foreign to the computer vision technologies. Heller is further foreign to the concept of combining the computer vision technologies with the RFID for analyzing the customers' behavior and associating the demographics and transaction data of the customers to the RFID, especially in a shopping environment.
U.S. Pat. No. 6,724,308 of Nicholson disclosed a method for multi-functional RFID tag assemblies, passive repeater systems, and modular antenna systems to track containers and products. With this approach, Nicholson tried to overcome the weaknesses in the RFID technologies, such as the disruption of the communication of the antenna with the RFID tags due to the objects that pass through the antenna field range. Nicholson is clearly foreign to the idea of associating the RFID technology with another technology, such as the computer vision technology, for analyzing the customers' shopping behaviors. Nicholson is also clearly foreign to the idea of associating the RFID technology with the transaction data of the customers in a retail establishment. Nicholson is further foreign to the idea of tracking customers rather than tracking only containers and products. This is partially due to the fact that it is inconvenient and impractical to attach the RFID tags to the customers and rely only on the RFID tags for the customers' tracking, which otherwise could be handled in an easier and more practical manner in the computer vision technology applications using the visual images of the customers.
U.S. Pat. No. 6,791,603 of Lazo, et al. (hereinafter Lazo) disclosed a system for event driven video tracking system, where the RFID is used as an event initiation method. In Lazo, the sensed event triggered the video tracking of the asset that was associated with the RFID tag, from zone to zone, within a controlled location in a surveillance application. Lazo did not explicitly describe any method for an automatic video tracking using a computer vision based tracking technology. Lazo is also foreign to the automatic demographic analysis of the customers using the visual images of the customers. Therefore, Lazo cannot explain about the novel usage of computer vision technology for analyzing shopping behavior and demographic composition of the customers in a retail establishment in association with the RFID tracking information. Lazo is further foreign to the concept of associating the transaction data in the retail environment to the customers' shopping behavior based on the tracking information by the RFID and the computer vision technology.
U.S. Pat. No. 6,998,987 of Lin disclosed a surveillance system that integrates RFID and video tracking. Although Lin briefly mentioned the video tracker identifies objects within scenes based on a variety of visual features, such as a group of pixels having a particular aspect ratio, corresponding to the color and composition of a typical face, or corresponding to the structure of lines or corners of an object, Lin did not explicitly describe how the video tracking can be used for shopping behavior analysis of shoppers or customers in a retail environment in association with the RFID information. Lin is further foreign to the concept of associating RFID tracking information with the demographic composition of the customers in a retail establishment or the transaction data of the customers in the retail environment utilizing the tracking information by the RFID and the computer vision technology.
The prior arts are also foreign to the concept of correlating the multiple associations of the RFID based tracking, the video-based analysis, and other temporal data, without using a large number of means for capturing images. The present invention can correlate the plurality of associations and provide an entire shopping path analysis of a customer according to the time sequence, and the correlation process can reduce the cost for hardware in a store by installing the means for capturing images in selected locations rather than installing them throughout the entire store.